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ABSTRACT 

More than 10,000 instructional software packages have 
been published for elementary and secondary schools — thousands in 
each discipline. Unfortunately, not all sources of courseware 
evaluations are equally reliable; therefore, it is important to 
determine that evaluations include: (1) critical appraisal of content 
accuracy; (2) in-depth consideration of the appropriateness, and 
effectiveness of the instructional strategies employed; and (3) input 
from testing with students. Some comprehensive and accessible sources 
of courseware evaluations are Micros I FT Courseware Evaluations, which 
probably contain the most thorough and consistently reliable 
evaluations, and EPIE Micro-Courseware PRO/FILES, whose evaluations 
are synthesized from the reviews of two or more evaluators who have 
usually tested the courseware with students. Periodicals which 
contain software evaluations include the bimonthly "Electronic 
Learning," and the monthly "Educational Technology." Two sources that 
identify only high quility courseware (although according to 
different criteria) are "Only the Best, the Discriminating Software 
Guide for Preschool-Grade 12" (L. L. Mattas) and "The Yellow Book: A 
Parent's Guide to Educationally Sound Courseware" (NBA Educational 
Computer Service). (JB) 
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MICROCOMPUTER COURSEWARE EVALUATION 

SOURCES 



The Dilemma: Too Much Choice 

More than 10,000 instructional software packages have b°en 
published for elementary and secondary schools— thousands in each 
major discipline. We know, from magazine and journal laments, 
that much of this courseware is of poor quality. Faced with such 
an overwhelming supply, how can you increase the probability of 
purchasing good courseware for your school system? 

The Solution: Courseware Evaluation 

Fortunately, others have addressed this problem and have im- 
plemented a solution: evaluating large numbers of courseware 
packages. Thus, your school system can begin its task by identify- 
ing potentially suitable courseware through the evaluations available 
from one or more evaluation projects. After verifying that the con- 
tent of some of the courseware is appropriate to your needs, you 
can select these few programs for your own specialists to evaluate 
prior to purchase. 

Some Cautions 

Not all sources of courseware evaluations are equally reliable. 
Before depending on the conclusions reached by a particular evalua- 
tion service, learn what you can about the evaluation methods and 
criteria it employs. It is especially important that evaluations 
include: 

• critical appraisal of content accuracy, » 

• in-depth consideration of the appropriateness and effectiveness 
of the instructional strategies employed, and 

• input from testing with students. 

Note, too, that some evaluation services also consider how closely 
a given courseware package matches a state or local curriculum. 
This selection phase may not yield conclusions that are valid for 
your school system. 

Useful Sources of Courseware Evaluations 

Identified in this Digest are just a few of the most comprehen- 
sive and readily accessible sources of courseware evaluations. 
Described below are reviews available from two major courseware 
evaluation projects and those published in two magazines. Also 
highlighted are two sources that identify only high quality 
courseware (according to different criteria). Courseware evaluations 
are also available through clearinghouses established by several 
states and provinces, including Alberta. British Columbia, Califor- 
nia, Florida, Iowa, Maryland, Minnesota, New York, North 
Carolina, Texas, and Utah. 

O 



MicroSIFT Courseware Evaluations, Holznagel. D.C., & Weaver. 
D. W. (Eds.). (1982-1985). Courseware evaluations, sets 1-16. Portland 
OR: Northwest Regional Educational Laboratory. (ERIC Document 
Reproduction Service Nos. ED 226 765 (sets 1-5); ED 234 772 (sets 
6-8); ED 239 606 (set 9); ED 245 666 (set 10); ED 249 918 (sets 1 1-12): 
ED 260 710 (sets 13-14); see the April 1986 issue of Resources in 
Education (RIE) for the ED numbei for sets 15 and 16. 



Probably the most thorough and consistently reliable evaluations 
are published by the MicroSIFT project. Each courseware package 
is reviewed by at least three professionals and tested with students 
from the intended audience group. 

The synthesized evaluations include descriptions of the 
courseware (objectives, prerequisites, content and structure, 
documentation, potential uses), appraisals of major strengths and 
weaknesses, and a check against more than 20 content, instruc- 
tional, and technical criteria. The evaluation concludes with a sum- 
mary of the content; instructional and technical ratings: and an 
overall recommendation statement (highly recommend, recom- 
mend use with little or no change, recommend use only if certain 
changes are made, or do not recommend). A modified format has 
been developed for science courseware that includes descriptions 
of science processes and concepts and three additional content and 
instructional criteria. 

These evalutions (issued periodically in sets of 75 to 90) may be 
obtained fronr the ERIC Document Reproduction Service (call 
1-800-227-3742, prices vary), or through the online Resources in 
Computer Education (RICE) database available through BRS (call 
MicroSIFT: 503-248-6800 Ext. 551). Some of the later sets contain 
evaluations of courseware that has been revised since an earlier 
MicroSIFT evaluation was published. Sets 15 and 16 will be the 
last sets of evaluations to be published by the MicroSIFT project 
in the current form, but the project will continue to collect evalua- 
tions and make them available through the RICE database 



EPIE Micro-Courseware PRO/FILES. (1982-1985). Microcomputer 
courseware pro/files & evaluations. Water Mill, NY. EPIE Institute. 
Available by subscription. 



A comprehensive evaluation service is provided by the Educa- 
tional Product Information Exchange (EPIE) in cooperation with 
Consumers Union (CU). Each Courseware PRO/FILE is synthesiz- 
ed from the reviews of two or more evaluators. who have usually 
tested the courseware with students. 



Every 3- or 4-page Micro-Courseware PRO/FILE and Evaluation 
contains a lengthy description and evaluation of content, teacher 
and student use, and management, often including photos of sam- 
ple screens. Instructional design and software design are also ap- 
praised under the general headings of goals and objectives, con- 
tents, methods and approach, and evaluation and management. A 
summary of the evaluation is provided on the first page, along with 
recommendations to the producer. These are accompanied by 
overall ratings of instructional design and software design (in earlier 
PRO/FILES] or a summary recommendation (in more recent 
PRO/FILES). 

Also published by EPIE Institute and Teachers College Press. Col- 
umbia University, is TESS: The Educational Software Selector (1985 
Edition, S59.95). This guide includes review citations and an in- 
dication of whether a review is favorable. 



Electronic Learning. Burroughs. R. (Ed.). New York: Scholastic. 
Published monthly September, October, and January through April; 
published bimonthly November/December and May/June. 

Each issue of this magazine for teachers includes an entire sec- 
tion on the instructional applications of computers. In addition to 
discussing how computers can be used to help teach, for example, 
foreign languages or business education, contributors to that sec- 
tion provide comparative evaluations of several relevant courseware 
packages. The review's, completed by a single evaluator, are not 
detailed, but do provide a concise overview of each program's 
strengths and weaknesses, along with seven ratings (instructional 
design, content, appropriateness, interest level, ease of use, sup- 
port materials, and overall value). Also included are the publisher s 
comments on the evaluations, if available. 



Educational Technology. Lipsitz, L. (Ed.). Englewood Cliffs. NJ: 
Educational Technology. Published monthly. 



Every issue of this well-respected publication includes reviews 
of two or more courseware packages. Each evaluation is provided 
by a single professional, who judges the program against explicit 
criteria and who is required to test the courseware with one or more 



students representative of the intended audience The prose evalua 
tion reports consist of a detailed description of the courseware 
package and a critical evaluation of its strengths, weaknesses, and 
potential use. Publishers are given an opportunity to respond to 
the reviews. 



Mattas. L. L. (1985). Only the best: The discriminating software guide 
for preschool-grade 12. Sacramento. CA: Education News Service. 
ERIC Document Reproduction Service No. ED 256 294. 



For those who do not have the time to read hundreds of individual 
evaluations, Only the Best is an excellent resource. In it. Mattas has 
identified 113 programs that received the most agreement on high 
quality among the 16 evaluation services whose courseware reviews 
she examines. A one-page report is provided for each program. 

Also included in Only the Best are brief descriptions of 189 nearly 
qualifying" programs and 13 more that Mattas considers "worth 
looking at." (For courseware in these last two groups, you would 
certainly want to read one or two more thorough reviews before 
making a decision to select any of the packages.) 



NEA Educational Computer Service (1985). The yellow book: A 
parent's guide to educationally sound courseware. Washington. DC: 
National Education Association. 



Teachers/reviewers of the NEA Educational Computer Service 
examined 1,500 courseware packages from which they identified 
as being of high quality, the 272 programs described in The Yellow 
Book. The descriptions provided for each package are brief, giving 
an overview of the content and a summary of the evaluator's com- 
ments and recommendations. Again, you would probably want to 
conduct your own evaluation of any of these packages you consider. 

Also identified in this publication are subsets of the 272 programs 
that are available in versions for specific microcomputers, including 
many less popular models. 



This digest was prepared by Robin Taylor. Coordinator of Com- 
puter Education. University of Maryland Baltimore County. 
Catonsville. MD 21228. December 1985. 
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